COHOMOLOGICALLY INDUCED DISTINGUISHED 
REPRESENTATIONS AND A NON- VANISHING HYPOTHESIS 
FOR ALGEBRAICITY OF CRITICAL L- VALUES 



BINYONG SUN 

Abstract. Let G be a real reductive group with an involution a on it. Let H be 
an open subgroup of G"^ with a character x on it. Associated to a "theta stable", 
"(T-split" parabolic subalgebra, and using the Zuckerman functor, we construct 
representations of G together with ^-equivariant linear functionals on them. We 
apply this construction to prove a non-vanishing hypothesis of H. Grobner and A. 
Raghuram in the study of algebraicity of critical L- values. 



1. Introduction 

For any positive integer n, write 112°'^ (R) for the set of isomorphism classes of 
irreducible Casselman-Wallach representations vr of GL2„(IR) such that 

• ^lsL±2n(iR) unitarizable and tempered, and 

• there is a representation F in GL2n(C) such that the total relative Lie algebra 
cohomology 

H*(g2„,i^2n;7r®F) ^0. 

Here g2„ := 0t2n(C), JsTan := S0(2n)M!^ C GL2n(K), and GL^^) is the set of 
isomorphism classes of irreducible algebraic finite-dimensional representations of the 
complex group GL2n(C) . Recall that a representation of a real reductive group is 
said to be Casselman-Wallach if it is Frechet, smooth, of moderate growth, and its 
underlying Harish- Chandra module is admissible and finitely generated. The reader 
may consult [Cas], [Wal, Chapter 11] or [BK] for more details about Casselman- 
Wallach representations. The usual notion of real reductive groups is explained 
in Section 3.1. To ease notation, we do not distinguish a representation with its 
underlying vector space, or an irreducible representation with its isomorphism class, 
or a one- dimensional representation with its corresponding character. 

The representations in 112°'^ (R) are real components of cuspidal automorphic rep- 
resentations of GL2n which are regular algebraic in the sense of Clozel ([Clo]). The 
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reader is referred to [Clo, Section 3] as a general reference for the following discussion 
concerning representations in n2°^(M) and their cohomology. See also [Mah, Section 
3.1], [RSI, Section 5], [GRl, Section 5.5] or [GR2, Section 3.4]. 

It follows from Vogan-Zuckerman's theory of cohomological representations ([VZ]) 
that 

n^°^(M) = {tti,^ I w e Z, 1 = (/i > /2 > • • • > /n > 0) e (w; + 1 + 2Z)"}, 

where 

(1) TTi,^ = TTi (g) |det| 2 , 
and 

TTi := Indp^ I lDi^®Di^® ■ ■ ■ ®Di^ (normalized smooth induction). 

Here P2,2,- -,2 C GL2„(M) is the block- wise up triangular group corresponding to the 

partition 2n = 2 + 2H 1-2, and Di. is the unique irreducible Casselman-Wallach 

representation of GL2(M) which is unitarizable, tempered, and has infinitesimal char- 
acter (^, ^). 

Denote by F\ _yj the representation in GL2„(C) of highest weight 

/V , ,W W WW. 

(2) A,-p2n-(2> 2'---' 2' 2^^^ ' 
where 

, _ Ji ^2 In —In —h 
' ^ 2"' 2"' " ' ' 2"' " ' ' ~Y' 
is the infinitesimal character of tti, and 

.2n-l 2n-3 1-1 3 - 2n 1 - 2n 

2' " ' 2 ' 2 ' 



P2n '■— { 7, , 7, , • , „ , 



is a half sum of positive roots. Then is the only representation in GL2n(C) so 
that 

(3) H*(02n, K^n] TTi,^ ® ^1,-^) 0. 

The highest degree of non- vanishing of (3) is 

Po -.^ + n — 1. 
Recall that the cohomology (3) is computed by the complex 

{ Homj^2n('^''(S2n/£2n), TTl,^ ® ] .^^ , 

with certain coboundary maps. Here and henceforth, we use the corresponding 
lower case Gothic letter to indicate the complexified Lie algebra of a Lie group. 
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(For example, is the complexified Lie algebra of K2n-) At the highest degree of 
non- vanishing, it turns out that 

(52n, K2n; TTj,^ ® Fi = Hom^^n (d2n/hn) , 7^l,w ® ^1 

(4) = HomdetKA^'°(02nA2n),7ri,^®i^l,-«;), 

egZ/2Z 

with each summand of (4) has dimension one. Here "Homdet^" indicates the space 
of linear maps / such that 

(5) fik.v) = det{kyk.{f{v)), k e 0(2n), v e A''°{92n/hn). 

Similar notation will be used later on without further explanation. 
Put 

1+ := (/i + l,/2 + l,--- ,/„ + !), 
and let ri+ be the corresponding irreducible representation of 0(2n), as in Section 2.1. 
It is easy to see that Ti+ occurs with multiplicity one in tti^^^ as the unique minimal 
0(2n)-typc in the sense of Vogan (cf. Remark 2.5). We fix an 0(2n)-equivariant 

embedding and view ri+ as a subspace of 7r\^w 

The key concrete result of this paper is the following 

Theorem A. Let vri^^ be a representation in Il2°^{M.) as in (1). Let X = Xi ® X2 be 
a character of GLn{M.) x GL„(]R) so that X1X2 = det"". Then the restriction to ti+ 
induces a linear isomorphism 

(6) HomGL„(R)xGL„(R)(7ri,^,x) = Homo(„)xO(n)(ri+,x), 
and both spaces in (6) have dimension one. 

Now we consider the finite-dimensional representation Fi^^w Put 

j_ r 1 - (2n - 1, 2n - 3, ■ ■ ■ , 3, 1 + 1/2), if /„ = 1 and w/2 is odd; 
1 1 — (2n — 1, 2n — 3, • ■ ■ , 3, 1), otherwise, 

and write ti- for the corresponding irreducible representation of 02n(C), again as 
in Section 2.1. Then ti- occurs with multiplicity one in Fi^^yj as the unique max- 
imal 02n(C)-type (see Lemma 2.4 and Remark 2.5). We fix an 02n(C)-equivariant 
embedding and view ri- as a subspace of -Fi^_^. 

The finite-dimensional counterpart of Theorem A is 

Theorem B. Let Fi_yj he a finite- dimensional representation ci/GL2n(C) as in (2). 
Let wi,W2 e Z n [^—^1^, ^""2""* ] be two integers so that wi + W2 = —w. Then the 
restriction to Ti- induces a linear isomorphism 

(7) HomGL„(C)xGL„(c)(i^i,-.«, det-^'-^) - Homo„(c)x04C)(ri-, det"'^''-^), 
and both spaces in (7) have dimension one. 
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Here det"'^'"'^ denotes the character det^'^^det'^^ on GL„(C) xGL„(C), and similar 
notation will be used later on without further explanation. 

Remark. The set 1/2 + (Zn [ ^~'^~"' , ^"~^~'^ ]) consists precisely all critical points of 
the standard local L-function L(s,7ri^^) (cf. [GR2, Proposition 6.1.1]^. 

To formulate the non- vanishing hypothesis of the title, put 

f)2n :=0ln(C) xg[„(C), 

and 

C2n f)2n H = (On(C) X 0„(C)) C. 

It is an important coincidence that 

dim(f)2n/C2n) = Po- 

Write 

(8) 32n '■ f)2n/C2n ^ 02nA2n 

for the inclusion map. 

Let X = Xi (8) X2 be as in Theorem A, and let Wi, W2 be as in Theorem B. Define 
e^. e Z/2Z so that 

Xj|o(n) = det'^S i = l,2, 

and put 

eo := n — 1 + e^-^ + Wi = n — 1 + €y^^ + W2 E Z/2Z. 
Using Theorem A and B, we prove 

Theorem C. Let ip^ be a nonzero element of the left hand side of (6), and let (pwi,w2 
he a nonzero element of the left hand side of (7). Then the linear functional 

Hom(A^"'(02nA2n),7ri,^®i^i,-«,) ^ Hom(AW(f)2n/C2n),X®det"^'"'^), 

)0f0 AfO j2n 

does not vanish on the one- dimensional space 

(10) Homdet^o(AP"(02„/€2n) 

and does vanish on the one- dimensional space 

(11) Homd<,t.o+l(A^"(02n/t2n), Tri,w ® i^l ,-«,)• 

Here "Hom" in (9) indicates the space of all linear maps. Theorem C is a slight gen- 
eralizatoin of the non- vanishing hypothesis of H. Grobner and A. Raghuram in their 
study of algebraicity of critical L-values using Shahka models, see [GR2, Hypothesis 
6.6.2]. As pointed out by them, up to minor variations, such a hypothesis appears in 
many previous articles, see, for instance, Ash-Ginzburg [AG], Harris [Har], Kasten- 
Schmidt [KS], Kazhdan-Mazur-Schmidt [KMS], Mahnkopf [Mah], Raghuram [Rag] 
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and Schmidt [Schm] . We expect that the method of this paper also works for these 
variations. The reader is referred to the afore mentioned articles for the importance 
of these hypotheses in the study of algcbraicity of critical L-values. 

In Section 2, wc deal with finite-dimensional representation theory problems en- 
countered in this paper. In particular, we prove Theorem B, and prove Theorem C 
by assuming Theorem A. Section 3 is devoted to a general construction of cohomo- 
logically induced distinguished representations. This lies at the heart of the paper 
and is interesting in itself. As an application of this construction, we prove Theorem 
A in Section 4. 

Acknowledgements: The author is grateful to Dihua Jiang for suggesting him to work 
on the non-vanishing hypothesis of Grobner and Raghuram. He thanks Patrick De- 
lorme for confirming the automatic continuity theorem in the twisted case (Theorem 
3.1), and thanks Hongyu He and Chen- Bo Zhu for helpful discussions. Part of the 
work was done when the author participated in the "Analysis on Lie groups" pro- 
gram at Max Planck Institute for Mathematics. The author thanks the organizers 
for invitation and thanks Max Planck Institute for Mathematics for their hospitality. 
The work was partially supported by NSFC grant 10931006. 



2. FiNITE-DIMENSIONAL DISTINGUISHED REPRESENTATIONS 

2.1. Irreducible representations of 02n(C). We start with a review of Cartan- 
Weyl's highest weight theory in the case of even orthogonal groups (cf. [GW, Section 
5.5.5]). Let K be a complex vector space of even dimension 2n > 2, equipped with 

a non-degenerate symmetric bilinear form on it. Denote by 0{V) the set of isomor- 
phism classes of irreducible algebraic finite-dimensional representations of 0{V). We 
intend to parameterize it by the set 

^0(2n) := { 1 = (/l > /2 > • • • > ^n) e Z^'^ X {-1/2,0, 1, 2, ■ ■ ■ } } . 

That is, for every 1 e Vo(2n), we shll define a representation ti G 0{V), and every 

representation in 0{V) is uniquely of the form ti. 

When n = 1, r_i/2 and Tq are defined to be the determinant character and the 
trivial character, respectively. Fix one of the two embcddings of in O(V^), for 
/ > 1, we define ti to be the two-dimensional representation of 0{V) which has 
weights ±1 when restricted to C^. 

In general, fix a fiag 



= (Xo C Xi C X2 C ■ • • C X,_i) 
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of totaly isotropic spaces in V so that dimXj — j, j — 0,1, - • • ,n — 1. Denote by 
Bjr the stabihzer of T in 0{V). Then we have an exact sequence 

l^U^^B^^ (CX)«-i X 0{Xti/Xn-i) ^ 1, 

where Ujr is the unipotent radical of Bj^, and the identification 

GL(Xi/Xo) X GL{X2/X,) X • • • X GL(X„_i/X„_2) = (C^)"-^ 

is used. Now for every 1 = (/i,/2,""- Jn) ^ 'Po{2n), we define ri to be the unique 
representation in 0{V) so that Tj^-^ descends to the irreducible representation 

of (CX)"~^ X 0(X^_^/X„_i). Here and henceforth, a superscript group (or Lie alge- 
bra) indicates the invariants of the group action (or the Lie algebra action, respec- 
tively) . 

As usual, we identify irreducible representations of the compact group 0(2n) with 
representations in 02n(C). 

2.2. Distinguished representations of 02n(C). For simphcity of notation, put 
:= 0„(C) X 0„(C). A shght modification of Helgason's proof of Cartan-Helgason 
Theorem show the following lemma (cf. [Hel, Chapter V, Theorem 4.1]). We should 
not go to the details. 

Lemma 2.1. Let ti G 02n(C) with 1 = {li, h^ - ■ ■ , In) G 'Po{2n), o.'n-d let ei, 62 G 'L/2'L. 
If one of the following three conditions 

(a) /„ > and h, h,-- ■ , /„ G ei 62, 

(b) In = 0, and k, h, - ■■ , /„-i G ei = 62 = 2Z, 

(c) In = -1/2, and h, h-,-' ■ 1 1n-i Gei-l = e2-l = 2Z, 

is satisfied, then 

dimHomo2(ri,det'i''2) = 1. 

Recall that every representation in 02n(C) is self dual. We should not provide a 
proof of the following elementary lemma. 

Lemma 2.2. Let r G O^ni'C) and let 61,62 G Z/2Z. If (f) is a nonzero element of 
Homga (r, def^'"^), then (p ® (p does not vanish on the one- dimensional space (r ® 

T-)02"(C). 
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2.3. Distinguished representations of GL2„(C). Denote by the irreducible 
algebraic finite-dimensional representation of GL2n(C) with highest weight 

(12) := (//I > //2 > • • • > //2n-l > l^2n) £ Z""- 

The following lemma is an instance of H. SchfichtkruU's generafization of Cartan- 
Helgason Theorem ([Sch2, Theorem 7.2]), see also [Kna, Theorem 2.1]). 

Lemma 2.3. Assume that 



A*l + A*2n — A*2 + A*2n-1 



for some integer w^. If wi,W2 & H \jJ>n+i, A*n] O'^d W1 + W2 — W/^, then 

dimHomGL„(c)xGL„(c)(i^M,det"""') = 1- 

2.4. The maximal 02n(C)-type. As usual, for any integer m > 0, we identify the 
group of algebraic characters on (C^)"* with Z"*, and write 61,62,- •• ,6^ for the 
standard basis of Z"*. 
Fix an embedding 

(13) 72„: (C")2"-^GL2„(C) 

of algebraic groups which sends (ai, 02, • • • , a2n) to the matrix 



ai+Q2n 












a2+a2n-i 
2 






0-271-1— 0-2 

2i 



a2n— Ql f) 

2i ^ 













2 2i, 
2i 2 















2i 




0-271-1+0-2 



2i 








n2n+ai 
2 



where i = a/— 1 G C is the fixed square root of —1. View (C^)^"" as a Cartan 
subgroup of GL2n(C) via the embedding (13). Then the corresponding root system 
is 

(14) {±(6i -ej)\l<i<j < 2n} C Z^". 

Fix a Borel subalgebra b2n of Q2n which corresponds to the positive system 

(15) {ei-ej\l<i<j < 2n} C Z^'^ 
of (14). Put 

(16) To := i2ni02n(C)) = {ia,,a2,- ■■ ,an,a-\--- ,a^\a^') e (C^)^^^} = (C><)^ 
and view it as a Cartan subgroup of 02„(C). Put 

bo := ban no2n(C). 
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This is the Borel subalgebra of 02n(C) corresponding to the positive system 

{ci ± Cj \ 1 < i < j < n} 

of the root system of 02n(C)- 

Let fjL be as in (12) and let e be a nonzero highest weigh vector with 
respect to b2n- Then it is also a highest weight vector with respect to bo- Therefore 
it generates an irreducible representation of 02n(C). It is easy to see that this 
representation is isomorphic to ri^, where 

(/il - /f^2n, /U2 - /"2n-l, ' " " , /"n - fhi+l - 1/2), if l^n = IS odd; 

(/ii - i22n, 1^2 - f^2n-i, ' ' ' , fhi - fhi+i) , otherwisc. 

Lemma 2.4. The irreducible representation ri^ of 02n(C) occurs with multiplicity 
one in F^. 

Proof. The lemma follows by noting that has weight 

[ill — ^2?i, ^^2 ~ A^2n-1, ■ ■ ■ 1 ^-n — ^-n+l) 

with respect to To, and this weight has multiplicity one in F^. □ 
Remark 2.5. As a slight modification of Vogan's definition ('[Vogl, Definition 5.1]^, 
we define the size of a representation ti G 02n(C) to be 



+ 2n - 2j7, 1 = (/i, /2, ■ ■ ■ , /n) e Po(2n). 

Then is ^/le unique maximal 02n{'C)-type of F^, namely, all other representations 
in 02n(C) which occur in F^ has strictly smaller size than ti^. 

2.5. Highest weight vectors in distinguished representations. Let K be a 

reductive linear algebraic group defined over C, with an algebraic involution a on it. 
Let C be an open subgroup of K'^. Let q be a cr-split parabolic subalgebra of the 
Lie algebra t of K, namely, q fl cr(q) is a Levi factor for both parabolic subalgebras 
q and cr(q). Denote by U the unipotent radical of the normahzer of q in K. 

As usual, we use "U" to indicate the universal enveloping algebra of a complex Lie 
algebra. The following lemma is useful to us. 

Lemma 2.6. Let F be an irreducible algebraic finite- dimensional representation ofl^ 
and let X '■ C ^ be an algebraic character. IfC meets every connected component 
o/K, then every nonzero element in Homc(F, x) does not vanish on F^ . 

Proof. Note that c + q = B since q is cr-split, where c denotes the Lie algebra of C. 
Therefore 

C.F^ D U(c).FU = U(c) U(q).FU = \]{t).F^ = Kq.F^, 
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where Kq is the identity connected component of K. Therefore 

C.F^ D C.(Ko.FU) = K.F^ = F, 

and the lemma easily follows. □ 

2.6. Proof of Theorem B. We use the notation of the Introduction in the remain- 
ing part of this section. It follows from Lemma 2.3 and 2.1 that both 

(17) HomGL4C)xGL„(C)(i^i,-«„ det'"^''^^) 
and 

(18) Homo„(c)xO„(C)(ri-,det'"^''^^) 

have dimension one. 
Write 

(19) (72„ : GL2n(C) GL2n(C) and a2n ■ 02n 02n 

for the conjugations by the diagonal matrix with the first n diagonal entries 1 and 
the last n diagonal entries —1. Note that the Borcl subalgebra of Section 2.4 is 
C2n-split. Let v^_^ G be a nonzero highest weigh vector with respect to b2n- 

By Lemma 2.6, a nonzero functional in (17) does not vanish on v^_^. It does not 
vanish on ti- since v^_^ e ti- by the argument of Section 2.4. This proves Theorem 
B. 

2.7. The space A^'°(02nA2n). Put 

bt := b2n n = bo © C. 

The space b2n/bj has dimension pq and A'^°{b2n/bt) has weight 

lo := (2n,2n-2,-- - ,2) 

with respect to the Cartan subgroup Tq of (16). Furthermore, A^°(b2n/b{) C 
A^°(02n/^2n) cousists of highest weight vectors with respect to bp. Therefore it gen- 
erates an irreducible representation of 02n(C) which is isomorphic to tiq. 

Lemma 2.7. The irreducible representation of 02n(C) occurs with multiplicity 
one in A*'° (£l2n/€2n) • 

Proof. Similar to the proof of Lemma 2.4, the lemma holds because that the weight 
lo occurs with multiplicity one in A*'°(02n/^2n)- D 

Fix a nonzero 02n(C)-equivariant linear map 

r]0 ■■ A^°(02n/^2n) ^ Tj^ . 

Recall the inclusion map j2n : f)2n/c2n Q2n/hn from (8). 
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Proposition 2.8. The map 

is nonzero. 

Proof. Identify 52n/^2n with the space S2n of symmetric trace free matrices in Q2n, 
and define on it an 02n(C)-invariant non-degenerate symmetric bihnear form 

{x,y) := tT{xy), x,y e S2n- 

This induces a symmetric bihnear form (, )a on /\^°{Q2n/^2n)- In order the prove 
the proposition, it suffices to show that the one-dimensional spaces /\^°{\)2nl <-2n) and 
A^°(b2n/bt) are not perpendicular to each other under the form ( , )a, or equivalently, 
the paring 

( , ) : f)2n/C2n X h2n/h ^ C 

is non-degenerate. 

Denote by a^^ : 02n/^^2n ^ Q2n/hn the map induced by a2n- Let x e f)2n/c2n so 
that 

(20) {x, b2n/b,) = {0}. 
Then 

(21) (X, <J°Jb2n/bO) = «(^), <(b2n/bO) = b2n/h) = {0}. 

Since (T2n(b2n) is opposite to b2n, we have 

(22) b2„/b« + a°„(b2n/bO = 02n/fi2n. 

We conclude that x = by combining (20), (21) and (22). This proves the proposi- 
tion. □ 

Lemma 2.9. The one dimension representation A*'°(f)2n/c2n) of 0„(C) x 0„(C) 
corresponds to the character det"~^ (g) det"""^. 

This is easy and we omit the proof. 

2.8. Proof of Theorem C. 

Lemma 2.10. Let e Z/2Z, i = 1,2, - ■ ■ ,7, and assume that 

ei + 63 + 65 = es + 64 + ee = e?- 
If (fo, V+ o-nd are respectively nonzero elements in 

Homo2(det'l•'^Tlo), Homo2 (ti+, det'=^''^) and Homo2 (tj-, det'^''®), 
then the map 

Homdet^7 (rio, Ti+ ^ Hom(det'^''^ det'=^''^ det'^''«). 
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is nonzero. 

Proof. Without loss of generality assume that £7 = 0. By taking the transpose of 
(po, the lemma is equivalent to saying that the map 

ipo ®^p-®(p^: ® Ti- ® Ti+ det'i'"' ® det''''« ® det'^''* 

does not vanish on {t\q®ti-®t\+)^^"^'^\ where is a nonzero element of HomQa (ri^, det"^'^^). 
Denote by v\q G t\q and v\_ e t\_ two nonzero highest weight vectors of weights 

lo = (2n,2n-2,--- ,2) and (Zi - (2n - 1), ^2 - (2n - 3), • • • , Z„ - 1), 

respectively, with respective to bo. Then v\q ®v\_ G t\q®t\- generates an irreducible 
representation t[+ of 02n(C) which is isomorphic to t\+. The functional ip'Q®(f_ does 
not vanish on t[+ since it docs not vanish on v\q ® v\_ by Lemma 2.6. Now apply 
Lemma 2.2 and we finish the proof. □ 

Assuming Theorem A, we are now ready to prove Theorem C. For the first asser- 
tion, it suffices to show that the map 

Homdet^o(rio,ri+ ®ri-) ^ Hom(Af«([)2n/c2n), X ® det"'^'"''^), 

) O / O (770 O Af0j2n) 

is nonzero. In view of Lemma 2.9 and 2.10, this is a consequence of Theorem A, 
Theorem B and Proposition 2.8. 

The second assertion holds because the image of (11) under the map (9) is con- 
tained in 

Hom,,,.o+i I , (A«'([)2n/C2n), X ® det^^'^^), 
and the later space clearly vanishes. 

3. COHOMOLOGICALLY INDUCED DISTINGUISHED REPRESENTATIONS 

This section may be read independently. 

3.1. Generalities on distinguished representations. Let G be a real reductive 
group, namely, it is a Lie group with the following properties: 

• the complexified Lie algebra g of G is reductive, 

• it has only finitely many connected components, 

• there is a connected closed subgroup of G with finite center whose complexi- 
fied Lie algebra equals to [5, g]. 

Let (T be a continuous involution on G and let H be an open subgroup of the cr-fixed 
point group G'^. 

Fix a Cartan involution of G which commutes a (for its existence, cf. [Ber]). 
Let q be a parabolic subalgcbra of g. We say that q is real cr-split if 

q = q and 6{q) = a(q). 
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and say that q is ^-stable cr-split if 

^^(q) = q and q = o-(q). 

Here a,9 : q ^ g arc respectively complexified difi^erentials of the involutions a, 9 : 
G ^ G, and ^ : g — )■ g is the complex conjugation with respect to the real Lie algebra 
of G. 

Fix a character x : i/ — )> C^. It is of general interest to construct Casselman- 
Wallach representations which map to the induced representation 

Indix {/ e Cr{G) I /(M = x{h)f{x), heH,xeG}. 

This is the same as constructing Casselman-Wallach representations vr of G, together 
with /J-equivariant continuous linear functionals (p : tt ^ x- We call such a pair 
(tt, If) a x-distinguished representation of G. 

One general way to get distinguished representations is the ordinary parabolic 
induction associated to real cr-split parabolic subalgebras of q. This is studied by 
many authors for trivial Xj see, for instance, Oshima-Sekiguchi [OS], Olafsson [Ola], 
Delorme [Del] , van den Ban [Ban] and Brylinski-Delorme [BrD] . On the other hand, 
Flensted- Jensen [FJ] and Oshima-Matsuki [OM] construct all discrete series repre- 
sentations on H\G, when they exist. SchhchtkruU [Schl] and Vogan [Vog2] prove 
that these are isomorphic to cohomologically induced representations associated to 
certain 6'-stable cr-split parabolic subalgebras, without describing the corresponding 
i7-invariant functionals. 

Note that if is a real reductive group and is stable under 9. Put K :— G^ 
and C := H^, which are maximal compact subgroups of G and H, respectively. 
The following automatic continuity theorem is due to van den Ban-Delorme [BaD, 
Theorem 1] and Brylinski-Delorme [BrD, Theorem 1] for trivial X) Delorme 
confirms to the author its validity in general. 

Theorem 3.1. Let E he a finitely generated admissible {q, K) -module. Then the 
restriction induces a linear isomorphism 

Hom^(E°°,x) = Hom^,c(^,x), 
where E°° denotes the Casselman-Wallach globalization of E. 

In view of Theorem 3.1, one may study distinguished representations in the set- 
ting of Harish-Chandra modules. We note that associated to a parabolic subalgebra 
which is either real cr-split or ^-stable cr-split, one may construct distinguished repre- 
sentations (in the setting of Harish-Chandra modules) by purely algebraic method. 
We carry out the construction in the ^-stable cr-split case in the remaining part of 
this section. The real cr-split case will be studied in more detail by the author and 
Chen-Bo Zhu in a paper in preparation. 
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3.2. Some subgroups and Lie subalgebras. Assume that q is a ^-stable cr-split 
parabolic subalgebra of g. We introduce the following diamond of groups 

(23) G' G 



H' K' ^ H K 



where 
and 



C C 
G' ■= Nciq) = NG(q) (the normalizer), 



H' := G' n H, K' := G' H K, G' := H' n K' . 

Denote by n the nilpotent radical of q fl [g, q\ . The parabolic subalgebras q and q 
decompose as 

q = g' ® n and q = g' ® il, 
and they are opposite to each other. Put 

qc := q n t and tic := n n I. 

Lemma 3.2. One has that 

(24) = + q = + q 
and 

(25) e = c + qc = c + qc. 

Proof. This is known to experts. We sketch a proof for completeness. Consider the 
real form 

00 := {a: e I (a o e){x) = x} 
of 0. Then q is real with respect to 0o, and is a Cartan involution. Therefore 

(26) = f) + q 

by the infinitesimal version of Langlands decomposition. Apply " ~ " to (26), we get 
= f) + q- 

The equalities (25) is proved in the same way. □ 
Lemma 3.3. One has that 

dim c/c' = dim tic = dim Uc — - dim t/t'. 
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Proof. We have 

c n qc = cr(c n qc) = c n qc = c n (qc n qc) = c n = c'. 
Therefore by (25), 

(27) c/c' = c/(c n qj = t/Ac = tie 

as vector spaces, which imphes the first equahty of the lemma. Other equahties are 
obvious. □ 

Write S for the value of the equahties of Lemma 3.3. 

3.3. Cohomological induction. Let E' be a finitely generated admissible (g', K')- 
module. View it as a (q, ir')-module through the trivial n-action. Then 

E° := U(0) ®u(q) E' 

is a (0, i^')-module, where q acts by left multiplication, and K' acts by the tensor 
product of its adjoint action on U(g) and its given action on E'. 

Denote by F^/^x the Zuckerman functor from the category of (5, ir')-modules to 
the category of {g, ii')-modules, and by F-^, ^ its derived functors, j — 0, 1, 2, • • • . 
We are concerned with the (g, iC) -module 

(28) £;:=Ff(£;'):=F|,^(£;°). 

As usual, we use "Ad" to indicate the adjoint action in variant contexts. Denote 
by C[K] the space of left i^-finite (or equivalently, right A'-finite) smooth functions 
on K. (Similar notation is used for other compact Lie groups.) In order to describe 
the module E more explicitly, we introduce an action of the quadruple 

(29) {t,K')x{Q,K) 
on the space 

(30) C[K] E°, 
as follows: 

• the pair (C, K') acts by the tensor product of the left translation on C[K] and 
the restriction of the {q, A'')-action on E°; 

• the group K acts on C[K] E° through the right translation on C[A']; 

• the Lie algebra g acts by 

(31) {X.f){k) := (Ad,X)./(A;), k e K, f e C[K] E° . 

In (31) and as usual, we identify <C[K] E° with a space of £^°-valued functions on 
K. 
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Under these actions, C[K]i^E° becomes a (fi, ir')-module as well as a weak (g, K)- 
module (cf. [KV, Chapter I, Section 5] for the notion of weak (g, il')-modules). Fur- 
thermore, the (t, /C')-action and the {g, i^)-action commute with each other. There- 
fore the relative Lie algebra cohomology group 

(32) W{t,K';C[K]^E''), j = 0,l,2---, 

carries a (g, fr)-action. It turns out that (32) is actually a [g, ir)-module (not only 
a weak (g, K)-modu\e) , and is canonically isomorphic to the derived functor module 
T^^, j^{E°) (cf. [DV], see also [MP, Theorem 1.6]). 

3.4. x-equivariant functionals. Let C act on A'^(c/c') by the adjoint action, and 
let I) act on it trivially. Then A"^(c/c') is an (t), C")-module. Write 

X ■■= A^(c/c')®X, 

to be viewed as a tensor product (f), C") -module. Now assume that we are given an 
([)', C")-equivariant linear map 

and we shall construct an (f), C)-equivariant functional </?:£'—> x in what follows. 

Lemma 3.4. There is a unique {^,C')-equivariant linear map (f° : E° ^ x° which 
eoctends </?'. 

Proof. We have that 

f) n q = (7(f) n q) = f) n q = ^ n (q n q) = f) n fl' = f)'. 

Combining with (24), we get 

E° = U(0) (g)u(q) E' = U(f)) ®u(t,') E' 

as an (f), C")-module. Therefore the lemma is a form of Probenious reciprocity. □ 

Similar to the action of the quadruple (29) on C[i^] ® E°, based on the (f),C")- 
action on x°, we define an action of the quadruple 

(33) {c,C')xH),C) 
on the space 

C[C] ® x°- 

Note that the quadruple (33) is component- wise contained in the quadruple (29), 
and the map 

(34) rK,c <H) 'P° : C[K] ^ E° ^ C[C] (g) x° 

is (c, C) X (f), C)-equivariant. Here rK,c is the restriction map, and (fi° is as in Lemma 
3.4. 

The following lemma is routine to check. 
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Lemma 3.5. Let the pairs (c, C) and (f), C) act on x° — A^{c/c') (8) x through their 
actions on A^{c/c') and x, respectively. Then the linear map 

^ ' f ^ JcXic)-'fic)dc, 

is (c, C") X {i),C)-equivariant, where do is the normalized Haar measure on C, and 
as usual, C[C] ® x° ^■^ viewed as a space of x° -valued functions on C. 

View x° ^ ^ (c, C') X ([), C)-module as in Lemma 3.5, then we have 

Lemma 3.6. One has an identification 

(36) H^(c,C";x°)=X 
o/ ([), C) -modules. 

Proof. Note that 

H^(c,C"; A^(c/c')) = Homc'(A^(c/cO, A^(c/c')) = C 
as a vector space. Therefore 

H^(c, C"; x°) = H^(c, C"; A^(c/cO) ®x = C®x = X- 

□ 

Restriction of cohomology yields an (I), C)-equivariant hnear map 

(37) E = H^(fi, K'- C[K\ ® E°) H^(c, C"; C[X] (g) £;°), 

and the composition of (34) and (35) yields an (1), C)-equivariant linear map 

(38) H^(c, C- C[K] ® E°) ^ H^(c, C"; x°) = X- 
Finally we obtain the desired ((), C)-equivariant linear map 

(39) ^ := m^') -.E^x 
by composing (37) and (38). 

Remark. It seems that meromorphic continuation is needed in order to construc- 
tion general x-equivariant functionals on r|-, ;^(Homu(q)(U(5), i?')^/). The later is 
another form of cohomologically induction which is isomorphic to E when E' lies 
in a "general position" . This is similar to the construction of H -invariant linear 
functionals on the usual parabolically induced representations, where meromorphic 
continuation is used in the literature. 
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3.5. Bottom layers. Let F be an irreducible representation of K. The matrix 
coefficient map 

mp-.F^F* ^ C[K], 

u®v I—)- {k v{k.u)) 

is Kx K-equivariant, with the first factor oi KxK acts on C[K] by right translations, 
and the second one acts by left translations. We have a X-space containment 

(40) E = }l^{t,K';C[K]^E°) D R''^ {t K'; F (g) F* ^ E°) = F ^W'^ {t, K'; F* (g) E°). 

Let F' be an irreducible representation of K' which is i^'-equivariantly embedded 
in E'. Assume that we are given a X'-equivariant linear map 

■ A^Uc ^ {F* f^ ® F'. 

It induces a map 

■■ A^tlc ^ F*<^E° 
since C F* and F' C E' C E° ^ \J{q) (8)u(q) E'. 

The following lemma is essentially known and routine to check. 

Lemma 3.7. The composition map 

is a cocyde of degree S in the complex { Hom/^/(A-'(t/t'), F* E°) which com- 
putes the cohomology group I{^{t,K'-F*®E°). Here 

: ^ = tie 

is the quotient map. 

The map (3q of the above lemma represents an element [/3o] of H'^(€, K'] F* (8) E°). 
Combining this with (40), we get a (i^'-equivariant) bottom layer map 

3.6. Non- vanishing of cp on Bottom layers. Recall that we are given an (f)', C')- 
equivariant linear map 

ip':E'^X° = A'ic/c')0X- 
By tensoring with (F*)'^=, its restriction to F' yields a linear map 

■■ {F*y- ^ F' ^ {F*y^ (g x° ■ 

The image of the composition map 

(41) <^fo/3o: A^n,^(F*)«=0x° 

is at most one-dimensional and is therefore of the form vp* ®x° for some vp* e (F*)"'=. 
We have the following criteria of non-vanishing of on bottom layers: 



18 BINYONG SUN 

Proposition 3.8. The functional (p o f3 on F is nonzero if and only if 

x(c) c.vp* dc ^ 0. 

Proof. Let v e F. Then /3{v) e E is represented by the cocycle 

v^Po^ HomK'iA^ii/i'), F ® F* ® E°) C RomK'{A^ (t/t'), C[K] ® E°). 
Recall the map 

rK,c ® ^° : C[K] ® E° ^ C[C] ® x° 

form (34), and write 

J, : c/c' ^ t/t' 

for the inclusion map. Denote by a generator of the image of the composition of 

A^(c/c') ^ A^(B/B') C[i^] ® F° C[C] ® x°, 

to be viewed as a x°-valued function on C. Then by the construction of (p, we know 
that ip{(3{v)) = if and only if 

(42) / x(c)-'&.(c)dc = 0. 

Jc 

Note that the map v <S> is the composition of 
Since the composition of 

A^(c/c') ^ A^(e/e') "-^^ A^n. 
is a linear isomorphism, by a generator of the image of the composition of 

A5(n,) ^-^ C[K] ® C[C] ® x°- 

This composition map is the same as the composition of 

(43) A^(n,) '-^ F ® F* E° F F* x° C[C] ® x°- 

By the definition of vp*, the image of the composition of the first two maps in (43) 
is V <S) vp* ^ X° ■ Hence (42) is equivalent to 

/ x{c)~^ vf*{c.v) dc — 0, 
Jc 

and the proposition follows. 

□ 
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Remark. The argument of this section goes through if we replace q by an arbitrary 
6-stable parabolic subalgebra of g (with the assumption that q fl q is a common Levi 
factor of q and q, as usual), and replace H by an arbitrary 6-stable closed subgroup of 
G so that q + i) = Q- This should provide a construction of discrete series for general 
spherical reductive homogeneous spaces. We hope to study this in a future work. 

4. Proof of Theorem A 

We continue with the notation of last section and specify the discussion to the 
following case: 

• the group G — GL2n(IR); 

• the involution a = (see (19)), 

• the Cartan involution 9 is the inverse transpose of matrices, 

• the parabohc subalgebra q = b2n (see (15)), which is ^-stable cr-spht. 

Identify G' with the group 

(C^)" = {(ai, as, • • • , a„, a„, ■ • • , as, ai) G (C^)^"} 

through the embedding 72„ (see (13)). It is a fundamental Cartan subgroup of 
GL2n(K)- The groups in (23) become 

(Cx)" GL2„(M) 



(§^)" ^ (GL„(R))2 0(2n) 



{±1}" (0(n))2 

Here §^ is the group of complex numbers of modulus one. 

We also use the notation of the introduction. Let E' be the one- dimensional 
algebraic representation of 72n((Cx)^") corresponding to the character 



View it as a representation of G' through restriction, and we form the (g, K)-m.odu\e 

E :^ rf (E') 

as in last section. Here S := \ dim fic/^c = n'^ — n. 



Write F' :— E', viewed as a representation of K' = (§^)". It has weight 

(/i + 2n- l,/2 + 2n-3,-- - ,/n + l)- 
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Recall the representation ti+ of 0(2n) from the introduction. Put F :— ti+. Then 
F* = F and has dimension two, with weights 

(-Zi - 1, -Z2 - 1, • • • , -^n - 1) and (-Zi - 1, -^2 - 1, • • • + 1)- 

Note the one-dimensional representation A'^tic of K' has weight 

(2n-2,2n-4,-- - ,0). 

Fix a nonzero iC'-equivariant linear map 

^0 : A^n, (F*y^ (8) F', 

which exists and is unique up to scalar multiplications. We form the bottom layer 
map 

p:F^E 

as in last section. 

Recall from Theorem A that we are given a character 

X = xi®X2-H = GL„(M) X GL„(M) ^ 

such that 

(44) xiX2 = ^ei''. 

Note that A'^(c/c') is trivial as an (f), C") -module, and (44) ensures that 

x\h' — E'\h', 

both correspond to the character 

H' = (RX)" ^ C^ (ai, 02, • • • , an) ^ (0102 a„)"'. 

Therefore E' is isomorphic to x° — /^'^(c/c') ® X as an if'-representation. Fix an 
iif'-equivariant linear isomorphism 

As in last section, we construct an (f), C)-equivariant linear map 

(p:E^X- 

Note that vp* (see (41)) is a nonzero element in (F*)"= of weight 
Lemma 4.1. One has that 

x(c) c.vp* dc 7^ 0. 

Proof. View F* as an irreducible representation of the real form 0(n, n) of 02n(C). 
Then a slight modification of Helgason's original proof of Cartan-Helgason Theorem 
shows the Lemma (cf. [Hel, Chapter V, Theorem 4.1]). □ 
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Now Proposition 3.8 implies that 

(45) (^0/3^0. 

The following propostition is essentially an instance of [VZ, Proposition 6.1]. We 
should not go to the details. 

Proposition 4.2. The underlying [q, K) -module of tti^^ is isomorphic to E. 

In view of Proposition 4.2, and apply the automatic continuity theorem (Theorem 
3.1), we obtain a GL„(M) x GL„(M)-equivariant continuous linear map 

which does not vanish on the minimal 0(2n)-type ri+. 

We finish the proof of Theorem A by applying the following multiplicity one the- 
orem. It is proved by Aizenbud and Gourevitch when Xv,q is trivial ([AGS, Theorem 
8.2.4]), and their proof actually works in the general case. 

Theorem 4.3. Letp,q > 0. Let Hp^q be an irreducible Casselman-Wallach represen- 
tation of GLp^qiM.) and let Xp,q be a character on GLp(M) x GL^(M), then 

dimHomGLp(R)xGL,(R)(7rp,g,Xp,g) < 1. 
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